Voice communications with real-time status notifications

ABSTRACT

A system, apparatus, graphical user interface and methods are provided for conducting electronic voice communications with status notifications. A user of a first portable device executing a communication application selects a multi-function control that automatically initiates an audio recording, which is automatically sent toward another participant of an active communication session when the control is released. The same multi-function control may be used to transmit a textual message. A representation of the recording is displayed on the user&#39;s device, with an indicator that changes appearance when the other participant plays the recording. In addition, when one user initiates an audio recording that will be automatically sent to another user, the other user&#39;s device is advised of the initiation of recording and displays that status for the other user.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of and claims the benefit of priorityto U.S. patent application Ser. No. 13/958,408, titled “VoiceCommunications with Real-Time Status Notifications,” filed on Aug. 2,2013, which is hereby incorporated by reference in its entirety.

BACKGROUND

This invention relates to the fields of computers and electroniccommunications. More particularly, a system, apparatus, graphical userinterface and methods are provided for conducting electronic voicecommunications with status notifications.

Wireless telephones have become indispensable accessories and accompanymost people wherever they go. Mobile telephone users are thereforealmost always reachable via some form of communication. The ability toconduct live, real-time voice communications is still the main reasonmany people carry their phones, but many other people may use theirtelephones more for non-voice communications than for real-time voicecommunication.

In particular, many people use wireless telephones to exchange shorttext messages via SMS (Short Message Service). SMS messages aregenerally limited to 140 characters of text, and are conveyed viacontrol channels of wireless telephone service providers' voicenetworks. Thus, whether a wireless telephone user is sending a textmessage to one person or to many people, he or she is limited in howmuch he or she can communicate at one time. MMS (Multimedia MessagingService) was developed to allow media (e.g., images, video) to beconveyed to and from wireless telephones, but still relies on the SMSprotocol to deliver a notification to the recipient.

A wireless telephone user's ability to exchange recorded voicecommunications with another wireless telephone user, however, arecumbersome, even with MMS. In particular, initiating a voice recording(or other audio recording) on a typical wireless telephone requiresmultiple actions, such as manipulation of multiple controls in thecorrect sequence. For example, a user may need to open a recordingapplication, start a recording, end the recording, possibly name it forstorage, open a communication application, create a message to thedesired recipient, select an option to add an attachment, then find andattach the recording. Even if the originating and destination users areoperating a common communication application, the originator may stillneed to open a menu to find and select an option to make a recording,start the recording, stop it, then select an option to send it to therecipient.

SUMMARY

In some embodiments of the invention, a system, apparatus, graphicaluser interface and methods are provided for conducting voicecommunications with real-time status notifications. In theseembodiments, a user of one portable device is able to easily and quicklycreate and send an audio recording to a user of another device. Eitheror both users may receive real-time notifications of the status of theother user or of a recording.

In some embodiments, with a single manipulation of a multi-functioncontrol, a user of one portable electronic device creates and transmitsan audio recording to another user. The same control, in a differentcontext, allows the user to transmit a textual message separate from theaudio recording. In some implementations, the control may changeappearance depending on the context—such as whether or not the user hasentered text into a text window or dialog box.

In some embodiments, the sender of an audio recording receives areal-time notification that the recipient has initiated and/or completedplayback of the recording. In some implementations, a representation ofthe audio recording (e.g., in a graphically displayed version of theusers' conversation) or of the other user changes appearance to indicatethis change in status of the recording.

In some embodiments, the recipient of the audio recording may be advisedthat the sender has commenced an audio recording. For example, whenrecording begins, a signal may be issued toward the recipient's deviceand the recipient may see the status before the sender completes therecording.

DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram of a system for facilitating recorded voicecommunications with real-time status notifications, in accordance withsome embodiments of the invention.

FIG. 2 illustrates a user interface of a device for conducting voicecommunications with real-time status notifications, in accordance withsome embodiments of the invention.

FIGS. 3A-H illustrate a graphical user interface for conducting voicecommunications with real-time status notifications, in accordance withsome embodiments of the invention.

FIGS. 4A-E illustrate a graphical user interface for conducting voicecommunications with real-time status notifications, in accordance withsome embodiments of the invention.

FIG. 5 is a flow chart demonstrating a method of supporting voicecommunications with real-time status notifications, in accordance withsome embodiments of the invention.

FIG. 6 is a block diagram of a portable device for conducting recordedaudio communications with real-time status notifications, in accordancewith some embodiments of the invention.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the invention. Various modifications to thedisclosed embodiments will be readily apparent to those skilled in theart, and the general principles defined herein may be applied to otherembodiments and applications without departing from the scope of thepresent invention. Thus, the present invention is not intended to belimited to the embodiments shown.

In some embodiments of the invention, a system, apparatus, userinterface and methods are provided for conducting voice communicationswith real-time status notifications. In different embodiments, one ormore of multiple complementary features are implemented, such asone-touch voice recording, dynamic real-time notification to acommunication partner of the commencement of a voice recording, reliabledelivery of the recording to the partner, real-time notification ofplayback of the recording by the communication partner and automaticselection of a output device for playing an audio recording.

Embodiments of the invention are implemented within a communicationenvironment in which text, images, voice recordings and/or otherelectronic information are exchanged via data communication channelsconforming to an IEEE 802.11 standard (e.g., Wi-Fi® connections). Inthese embodiments, rather than relying upon traditional wireless voicecommunication channels that provide limited support for transmission ofinstant or real-time communications other than live voice conversations,the data communication channels allow unconstrained exchange ofvirtually all types of communications and media between devices of thesame and/or different types.

Illustrative embodiments of the invention are described as they may beimplemented as part of a one-on-one communication session (e.g., a chatsession), but may also be applicable to group communication sessions(e.g., group chat).

FIG. 1 is a block diagram of a system for facilitating recorded voicecommunications with real-time status notifications, according to someembodiments of the invention.

In these embodiments, users of communication system 120 operate avariety of portable communication devices operating the same ordifferent operating systems (e.g., Android, BlackBerry, iOS, Nokia S40,Symbian, Windows Phone). Thus, devices 102, 104 are operated bydifferent users, but may be configured similarly or very differently.Communication session 110 between devices 102, 104 comprises separateconnections between device 102 and system 120 and between device 104 andsystem 120.

System 120 is a data center, computer network or other cooperativecollection of computer resources for hosting a communication applicationor service (or a server portion of a communication application) that canbe simultaneously used by multiple users. In some implementations, theapplication provides a user with instant messaging of text, audiorecordings, photos, video and multimedia from his or her communicationdevice (e.g., device 102), to/from one or more other users operatingother communication devices (e.g., device 104).

The communication application hosted by system 120 is compatible withdevices 102, 104 and includes client logic that is executed under thedevices' native operating systems. A user operating one type of deviceand subscribing to one communication server provider can exchangemessages with users operating other types of devices and subscribing toother communication service providers. Individual messages may be sentto any number of recipients; for example, a group form of communication(e.g., chat) may be offered in which multiple users interactivelyexchange instant communications.

Communication servers 122 are front-end components of system 120 thathost users' connections with the system and that facilitate the exchangeof communications between users. Servers 122 may be referred to as“chat” servers, in that they receive real-time messages and chat sessioncommunications from individual users for delivery to one or more otherusers.

Illustratively, as long as a user is on-line with system 120 and theapplication is executing on his or her device, a connection or sessionmay be maintained between the device and a communication server 122.When the user closes the application on the device, his or herconnection may persist for some period of time (e.g., five minutes,fifteen minutes).

State servers 124 maintain states or statuses of users and/or users'devices. Thus, a state server stores some or all of a user's profile,including applicable preferences (e.g., preferred audible and/or visualalert, status message), device type/model, etc. Some status informationmaintained by a state server with regard to a user may persist only forthe duration of the user's connection with system 120. As the user'sdevice operates in different ways or different modes, goes offline,comes online and so on, a state server may capture those changes.

In some embodiments of the invention, however, statuses of users'devices or system connections are not retained. In these embodiments,however, a change in status of a user or user device may be reported tothe system by the device, and relayed to other users who are engaged incommunication (e.g., chat) sessions with that user, and/or to users whodo not have open communication sessions with the user but who havesubscribed to status updates regarding the user.

In these embodiments, as part of a user profile retained on a stateserver, identities of other users associated with that user may beidentified. These other users will be informed of some or all changes inthe user's status (or the changes in the status of the user'sdevice)—such as when she comes online and when she goes offline. Asdescribed below, other status changes may also be communicated to theassociated users.

Message servers 126 store communications that cannot immediately bedelivered to their target or destination users. A new message (e.g., atextual chat message) received for a user who is currently on-line withsystem 120 can be delivered immediately (i.e., through the user's openconnection with a communication server 122). However, a message receivedfor a user who is currently off-line (i.e., does not have an openconnection with a communication server 122) may be stored on a messageserver 126 until it can be delivered.

Media servers 128 store media files or content that await delivery to adestination user, other than messages that may be stored on messageservers 126. For example, a message from one user to another may be ormay comprise a photo, a video, an audio recording or some other media.The media is stored on a media server 128 until it can be delivered,which may be done automatically when the destination user is online, orwhich may be initiated only when the destination user requests the media(e.g., by clicking on a link in a textual message delivered to thedestination user to advise him of the media content).

Information identifying termination points of online users' connectionswith system 120 may be stored on communication/chat servers 122, stateservers 124 and/or other entities (e.g., a session server (not depictedin FIG. 1)). For example, each communication server may maintain a tableor other data structure identifying users whose connections with thesystem are terminated at or hosted by that server. Another entity (e.g.,a state server 124, a session server) may store a larger dataset thatidentifies, for every user connected to the system, which communicationserver maintains their connection.

Thus, when a communication server receives a new communication from aconnected user, for one or more other users, it may perform a lookupwithin local data and/or data stored on another system entity in orderto determine the other users' statuses (e.g., whether they areconnected, whether they are busy communicating) and, if so, where. Insome implementations, it will first search a local data table (e.g., arouting table), and only issue a query to another component if some orall destination users' statuses are not found locally.

Any number (e.g., one or more) of communication servers, state servers,message servers, media servers and/or other servers may be deployedwithin system 100. The functions of servers 122, 124, 126, 128 may bedistributed differently in other embodiments, such as by being mergedinto fewer servers or divided among additional servers.

The servers of system 120 may be separate physical computer systemscomprising hardware resources (e.g., processor, storage, communicationinterface) and software resources (e.g., program code, data) suitablefor performing the functions ascribed to them. In some alternativeembodiments, any or all of the servers may be virtual computing machinesthat share hardware and/or software resources. Thus, multiple softwareserver modules performing different functions may reside on one servercomputer.

Network(s) 150 comprise communication paths between devices 102, 104 andsystem 120, and may include data and/or voice communication channels.For carrying data communications, networks 150 may comprise theInternet, one or more intranets, LANs (Local Area Networks), WANs (WideArea Networks) and/or other point-to-point or distributed connections,which may be wired and/or wireless in nature. For carrying voicecommunications, network(s) 150 may comprise wired and/or wireless voicecommunication links managed by different telecommunication serviceproviders and implementing virtually any wireless technology, such asGlobal System for Mobile Communications (GSM), Enhanced Data Rates forGSM Evolution (EDGE), Long-Term Evolution (LTE), Code Division MultipleAccess (CDMA), Evolution-Data Optimized (EV-DO) and so on.

Networks 150 may thus comprise various entities not shown in FIG. 1,such as routers, gateways, switches, base stations (e.g., wirelessaccess points, Node Bs, Evolved Node Bs) and/or other equipment forestablishing, maintaining, switching or otherwise manipulating a dataand/or voice connection.

FIG. 2 illustrates a user interface of a device for conducting voicecommunications with real-time status notifications, according to someembodiments of the invention. In these embodiments, device 202 may be asmart phone or a portable computing device on which a communicationapplication hosted by system 120 of FIG. 1 has been installed.Therefore, an operator of device 202 is able to exchange communicationsin real-time with other users of the application. Their communicationsmay comprise text, images, audio recordings and/or other media. Device202 includes a touch-screen display, one or more microphones and one ormore speakers, and may include other components not referenced herein.

In FIG. 2, a user or operator of device 202 has opened the communicationapplication and is presented with user interface 204, including sessionthreads 210. Each session thread 210 pertains to a separatecommunication session with one or more users. Some or all threads mayinclude thumbnail images associated with another user involved in thesession, if that user has provided an image, such as thumbnail image 212a of thread 210 a. A thread may also identify the time or date of thelast communication exchanged with that user (e.g., time 214 a of thread210 a), and an excerpt of the last communication (if that communicationincluded textual content) or the full communication if it was shortenough (e.g., excerpt 216 a of thread 210 a.

A session thread will be listed for any active communication session theuser of device 202 is participating in, and also for any session thatthe user has participated in within some period of time that has passed(e.g., one week, one month, three months, one year), which may or maynot be configurable by the user. The user may need to scroll through thethreads if they cannot all be displayed at once on the display componentof device 202. Threads 210 may be sorted such that the thread associatedwith the most recently active communication session is at top, such thatthe most active session (e.g., the session with the greatest number ofmessages overall or greatest average communication rate) is at top, suchthat people selected by the user are listed first, or may be sorted insome other way.

Individual sessions among session threads 210 may be identified by thename of the other user participating in the communication session (forone-on-one sessions), or by the name of a group for group communicationsessions. A thread for a group session may also identify the member whosubmitted the most recent communication, may identify which members ofthe group are currently online and/or provide other information.

Besides session threads 210, interface 204 of FIG. 2 includes footer 220and header 240, either or both of which may provide soft controls forperforming various functions. Illustrative functions include search(e.g., to locate a thread associated with a particular user or group),contacts (e.g., to add a contact, to start a conversation/communicationthread with a contact), settings (e.g., to adjust an operating parameterof the communication application or the device, to modify an appearanceof the user interface), status (e.g., to set a status to be displayedfor communication partners and/or other users), creation of a new group,etc.

Hard controls of device 202 and other soft controls not depicted orlabeled in FIG. 2 may have other functions and may depend on the type ofdevice, the device's manufacturer, the device's operating system and/orother factors. These other functions are not listed here, so as not toobscure descriptions of features relevant to the invention.

FIGS. 3A-H demonstrate use of a graphical user interface for conductingvoice communications with real-time status notifications, according tosome embodiments of the invention.

In FIG. 3A, graphical user interface (GUI) 304 is displayed on thetouch-screen display of device 302 when the user selects a sessionthread 210 (e.g., thread 210 a) from interface 204 of FIG. 2. Userinterfaces 204, 304 may be different portions of the same interface.

Interface 304 features conversation region or area 310, which displaysrecent communications 312 sent as part of the current communicationsession. In particular, if the users previously exchanged anycommunications during this session or thread, those conversations (or atleast the most recent one(s)) will be displayed or represented inconversation region 310. Entire textual messages may be displayed (orjust initial portions of long messages), and representations of audiorecordings, images, videos and/or other communications that have beensent or received may also be displayed, as shown in subsequent figures.If the users have not conversed before (or within some period of time),or if this is a newly created conversation, region 310 may be devoid ofcommunications.

GUI 304 of FIG. 3A features footer 320 that, in the illustratedembodiment, includes multi-function control 322, text entry window 324and emoticon tool 326. Emoticon tool 326 may be activated to obtain apalette of emoticons for selection and entry into a text message. Textentry window 324 may be activated by the user to initiate or resumeentering a textual message.

Multi-function control 322 of GUI 304 operates in different manners,depending on the context in which it is presented and as furtherdiscussed below. In one context, such as when the user opens the currentcommunication session as shown in FIG. 3A, control 322 is displayed withone aspect or form—a microphone that can be selected (e.g., pressed,clicked).

In this context, the user can easily generate and send a voice or audiorecording to the other user(s) participating in the session.Specifically, by pressing and holding control 322, a microphone indevice 302 is activated and begins recording. Releasing the controlafterward causes the recording to be automatically transmitted towardthe other participant(s) in the conversation. In particular, therecording is transmitted to the system hosting the application (e.g.,system 120 of FIG. 1), and subsequently delivered to the recipient(s).In another context, and as discussed below, multi-function control 322enables transmission of a textual message.

Interface 304 of FIG. 3A also includes header 340 that offers thumbnailimage 342 of or associated with the other user participating in thecommunication session, if it is a one-on-one session, and return tool344 for returning to interface 204 of FIG. 2.

In FIG. 3B, graphical user interface 304 is displayed on thetouch-screen display component of device 302 as it may appear afterselection of multi-function control 322. In the illustratedimplementation, selection of the control includes pressing on it (e.g.,with a finger, a stylus or some other object) and holding. Uponselection of control 322, footer 320 of the GUI changes to include tip334, timer 336 and indicator 338.

As described above, selection of the multi-function control when it ispresented with the appearance shown in FIG. 3A automatically engagesmicrophone and audio recorder components of device 302. Initiation ofrecording may lag slightly (e.g., half a second, one second) in case ofaccidental or mistaken selection of control 322, to allow rapidabandonment of the recording process without sending anything tocommunication partner(s) 306.

Tip 334 serves to remind the user that he can cancel the recording bysliding horizontally across the footer with the object that is pressingcontrol 322. Timer 336 informs the user of the length of the recordingso far, and indicator 338 provides a visual clue that recording is inprogress. Indicator 338 may be colored (e.g., red), may flash orotherwise animate—to indicate recording is ongoing, to indicate arelative strength of the audio being captured by the device'smicrophone, etc. In some implementations, multi-function control 322takes on a slightly altered appearance when pressed, such as becomingdarker (or lighter) in hue.

FIG. 3C depicts GUI 304 of FIG. 3B after the operator of device 302releases multi-function control 322 and the interface returns to a statesimilar to that depicted in FIG. 3A. Communication 312 a has been addedto conversation area 310 to represent the audio recording. In addition,when the operator de-selected multi-function control 322 (e.g., byreleasing it), the recording was closed and the audio recording file wasuploaded to the communication system that hosts/supports thecommunication application associated with graphical user interface 304,for delivery to the recipient(s).

Communication 312 a may include various controls related to the audiorecording, such as a control for playing (and, alternately, pausing) therecording, a time field identifying a duration of the recording, aslider for rapidly accessing a desired portion of the recording (duringmanipulation of this control the time field may indicate a current timeposition within the recording) and/or others.

Also, however, communication 312 a is accompanied by thumbnail orrepresentation 314 a of the creator of the recording. In a one-on-onechat or communication session such as that depicted in FIG. 3C, eachcommunicant will usually know that a recording that appears inconversation region 310 and that he or she did not create must have beencreated by the other communicant. However, in a group communicationsession, sender representation 314 a may allow a group member to easilydetermine who originated the associated recording. In addition,representation 314 a may change over time based on a status of therecording (e.g., in addition to or instead of changes to the statusindicator described immediately below).

Communication 312 a is also accompanied by status indicator 316 a anddelivery indicator 318 a. Delivery indicator 318 a indicates whencommunication 312 a has been delivered to the other member(s) of thecommunication session. Illustratively, a single check mark (or othersymbol) indicates that the communication has been dispatched from device302 to the communication system. A pair of check marks (or other symbol)will indicate that the communication has been delivered to the otherparty's device, as shown for the previous communications 312.

Status indicator 316 a, however, will allow the operator of device 302to quickly determine whether the recipient has played the recording.When the recipient initiates playback of the recording, the recipient'sdevice will notify the system and the system will notify device 302,which will change or replace indicator 316 a. In some implementations,indicator 316 a changes color (e.g., from grey to blue); in otherimplementations, it changes in size or shape (e.g., to become a speakerinstead of a microphone). In some other implementations it disappears.In yet other implementations status indicator 316 a may change in otheror multiple ways, sender representation 314 a may change and/or otheralteration(s) may be made to communication 312 a to signify that therecording was played by the recipient.

In a group communication session, status indicator 316 a may change inappearance any number of times as group members play the recording(e.g., to indicate a number or percentage of recipients that have playedit), separate status indicators may be used for different members, theoperator may select the indicator to see a list of group members thathave (or have not) played the recording, etc.

Updates to status indicator 316 a (and/or other indicators and statusesdescribed herein) may or may not be reliable. In other words, even ifdevice 302 is offline or turned off when the recipient of the audiorecording associated with communication 312 a plays the recording,notification of that action may be buffered at the communication systemand delivered to device 302 so that the indicator can be updated whendevice 302 is again online.

FIG. 3D depicts GUI 304 of FIG. 3A when the operator or user of device302 selects text entry window 324 in order to compose a textual message.Conversation region 310 shrinks to accommodate appearance of keyboard350 for use in typing the message, and a cursor is placed in the textentry window. Multi-function control 322 retains its initial or firstappearance, and can be used as described above to create and send anaudio recording instead of, or in addition to, sending a textualmessage.

FIG. 3E depicts GUI 304 of FIG. 3D when the operator of device 302presses one or more keys on keyboard or keypad 350. In addition todisplaying in text entry window 324 the text corresponding to the user'sinput, multi-function control 322 changes to a second form or appearanceto indicate that it can now be used to send a textual message containingthe text within window 324.

In different embodiments of the invention, multi-function control 322may take virtually any forms or appearances (e.g., a symbol or symbolsin place of the word “Send”), and/or may offer different functionality,depending on the context in which it is displayed. In the illustratedexample, a first context is characterized by the absence of text enteredby an operator of the device, while a second context is characterized bythe presence of such text.

In some embodiments, however, even after a user has entered text intotext entry window 324, control 322 may still be usable to create anaudio recording, and may or may not change in appearance. In theseembodiments, briefly selecting control 322 (e.g., pressing and releasingit) may cause the text in the text entry window to be sent as a textualmessage, after which user interface may return to a state such as thatillustrated in FIG. 3A or 3D (with the new textual message displayed inregion 310).

However, in these embodiments, a longer selection of control 322 (e.g.,pressing and holding it) will cause recording to begin after apredetermined period of time (e.g., one second, two seconds). Releasingthe control will cause the recording to be forwarded toward therecipient, as described above, and may also cause the text in the textentry window to be sent as a textual message. Alternatively, after therecording is sent, the interface may return to the state it was in priorto generation of the audio recording (e.g., as illustrated in FIG. 3E)so that the user may complete his text entry, except that a newcommunication will appear in region 310 for the audio recording. Duringcreation of the recording in these embodiments, the user may still beable to cancel the recording, such as by sliding horizontally,vertically or in some other direction from control 322 while control 322is selected.

Returning to discussion of embodiments of the invention depicted in FIG.3E, if the user activates control 322 after composing a textual messagein window 324, that message will be sent and will be displayed (or arepresentation of the message will be displayed) in conversation region310. Keyboard 350 may close or may remain open to allow generation ofadditional text.

FIG. 3F depicts GUI 304 of FIG. 3D when the operator of device 302activates multi-function control 322 without pressing or before pressinga key on keyboard 350. As described in conjunction with FIG. 3D, audiorecording commences but can be cancelled as advised by tip 334.

FIG. 3G depicts GUI 304 of FIG. 3F after control 322 is de-selected andthe new audio recording is forwarded toward the recipient(s). Newcommunication 312 b is added to conversation region 310, and isaccompanied by or includes sender representation 314 b, status indicator316 b and delivery indicator 318 b, which serve similar purposes tothose discussed above in conjunction with FIG. 3C. In this case, thedouble check-marks of indicator 318 b may indicate that the audiorecording has been delivered to the recipient's device.

FIG. 3H depicts GUI 304 of FIG. 3C (or FIG. 3G) after the recipientcommences playback of communication 312 a (or communication 312 b) and areal-time notification of that action is transmitted to device 302. Inthese embodiments of the invention, status indicator 316 a changes colorto alert the operator of device 302 that his recording was played or isbeing played. The change in color (e.g., from grey to blue) may not beevident in FIG. 3H due to the black-and-white nature of the figures, butwill be notable in the color display of device 302. Alternatively, someother characteristic of status indicator 316 a may change (e.g., size,shape, fill).

FIGS. 4A-D illustrate a graphical user interface for conducting voicecommunications with real-time status notifications, according to someembodiments of the invention. In these embodiments, device 402 is asmart phone or portable computing device operated by the otherparticipant in the communication session reflected in FIGS. 3A-H—inother words, the recipient of an audio recording sent by the operator ofdevice 302. Device 402 may be similar to device 302, in that it includesa touch-screen display and a microphone; also, it executes the samecommunication application. It may or may not be the same type or modelof device or execute the same operating system.

Graphical user interface 404 displayed on the touch-screen display ofdevice 402 includes multi-function control 422, text entry window 424and emoticon tool 426, which may function as described above. Also,however, interface 404 includes status 448 of the other party to theconversation—currently “recording audio . . . ” in FIG. 4A. Thus, theoperator of device 302 is presently (or was very recently) recordingaudio to be sent toward device 402.

In the illustrated embodiments of the invention, when one member of acommunication session commences an audio recording (e.g., via amulti-function control provided herein), the other party is notified ofthat action in real-time. Prior to the change to status 448 to show thatthe operator of the other device is recording audio, the status may haveindicated that that operator was “online” or “last seen at <a particulartime>”, or “typing”, etc.

Status 448 may remain displayed until a different status is known. Forexample, when the operator of the other device completes the recording(and it is on its way to device 402), the status may be changedaccordingly. A device user may or may not be able to select a message orinformation to be displayed as status 448 on other user's device, orcreate a custom message.

FIG. 4B depicts GUI 404 of FIG. 4A when communication 412 a, an audiorecording that was being recorded by a communication partner of theoperator of device 402, is being downloaded. In some implementations,the recording may be downloaded automatically if/when device 402 isonline. In other implementations, the operator of device 402 may benotified that the recording is available (e.g., via a link displayed inconversation region 410 identified in FIG. 4A), and must take action todownload it (e.g., select the link). It may be noted that status 448 haschanged to reflect the other user's current status.

Communication 412 a may include symbols and/or controls for indicating astatus of the download, such as a slider that reflects the progress ofthe download, the duration of the recording, a control for canceling thedownload, etc. Communication 412 a also includes sender representation314 a and status indicator 416 a that indicates whether the recordinghas been played on the receiving device.

In some implementations, a status indicator such as indicators 316 a,416 a may differ in appearance between the sender and the recipient of arecording, and between different device platforms. For example, thestatus indicator may have different colors, different shapes, animationand so on. However it is presented, it will change in some way once therecipient begins or finishes playing the associated recording.

FIG. 4C depicts GUI 404 of FIG. 4B after downloading of the audiorecording represented by communication 412 a completes and it can beplayed (e.g., with a control embedded in communication 412 a). Statusindicator 416 a has not yet changed, because the operator of device 402has not yet played the recording.

In FIG. 4D, the operator of device 402 has played (or at least commencedplaying) the audio recording. Therefore, status indicator 416 a haschanged in appearance—in this case by changing color. In otherimplementations, indicator 416 a may undergo a more notable change, asdiscussed previously. The color change (from green to blue) reflected inFIG. 4D may not be apparent due to the black-and-white nature of thefigure.

FIG. 4E depicts GUI 404 of FIG. 4D when the user of device 402 selects(e.g., presses and holds) communication 412 a. When this action isdetected, menu 460 is displayed on the touch-screen display to allow theuser to take some action regarding the communication (and the associatedrecording). For example, as shown in FIG. 4E, options to delete orforward the recording may be offered. If the user wishes to respond tocommunication 412 a with her own recording, she may activatemulti-function control 422 as described above, and her recording will beautomatically created and sent with just the one touch on control 422.

FIG. 5 is a flow chart demonstrating a method of supporting voicecommunications with real-time status notifications, according to someembodiments of the invention. In the illustrated method, a user ofdevice 102 is engaged in a chat session with one other person—a user ofdevice 104. The illustrated method may be modified to support voicecommunications within a group communication session without exceedingthe scope of the invention.

In operation 500, the user of device 102 opens the communicationapplication on his device and opens (or creates) a conversation with theuser of device 104. He presses a multi-function control to initiate avoice or other audio recording (e.g., control 322 of FIG. 3A). Note thathe may also enter some text into a text entry window in association withthe audio recording, which may be delivered before the audio recording(e.g., if entered before commencing the recording), during (e.g., sentwith the recording) or afterward (e.g., if entered after completing therecording).

Upon initiation of the recording, the communication applicationexecuting on device 102 issues an ephemeral signal to communicationsystem 120. This signal is different from a normal chat message sent bythe application as part of a communication session, in that it is notdelivered to device 104 as a message. Instead, it is meant to beconsumed by the application logic as described herein.

In operation 502, the communication system receives the real-time statusnotification regarding commencement of a recording, and relays thatnotification to device 104. In some implementations, the notificationmay be retained on system 120 only long enough to relay it to device 104(or until acknowledgement of it is received from device 104), and thenit may be discarded. In other implementations, the status may beretained until it is replaced with a new status of device 102. Inembodiments of the invention reflected in FIG. 5, even if the user ofdevice 102 has active or open communication threads or sessions withother users, only the user involved in the current conversation (i.e.,the conversation for which the audio recording was initiated) isinformed of the commencement of the recording.

In operation 504, device 104 receives the notification of the audiorecording at device 102 and updates a display of the associatedconversation to indicate the status. Illustratively, the view of theconversation at device 104 may be altered to show “recording audio” or“recording” or some other indication of the action being taken at device102.

If the conversation is not currently being displayed on device 104, butthe communication application is running, the status notification may ormay not be shown. For example, if device 104 is currently displayingthumbnails of multiple conversations (e.g., as shown in FIG. 2), thethumbnail (or an icon or other portion of the thumbnail) of theconversation may blink, change color or otherwise take on an alteredappearance to indicate audio activity associated with that conversation.Or, if device 104 is currently displaying another conversation, a headeror other portion of that conversation may briefly change to identify theuser of device 102 and indicate that he is recording audio (e.g., with aprinted notification, with a tone, with an icon). For example, a tickerdisplay may scroll across the header, a portion of it may briefly changeto display the other conversation's status, the other user's thumbnailmay be displayed with an audible alert, etc. If, however, the user ofdevice 104 is currently recording or playing back audio from anotherconversation, no audible alert may be sounded, or there may be noindication at all of the status of device 102.

If the communication application is not currently executing on device104, the notification may be discarded. If the application is running inthe background, the notification may be consumed, but the user of device104 will only see it if she brings the application to the foreground.

In operation 510, the user of device 102 records his audio (e.g., voice,music, a sound effect) and releases the multi-function control. Uponrelease of the control, device 102 (i.e., the communication application)issues a new ephemeral signal to system 120 to report the change instatus.

In operation 512, communication system 120 receives the notification andprocesses it similar to the notification of operation 502, and relays itto device 104.

In operation 514, device 104 receives the notification and updates itsdisplay if possible (e.g., if the application is operating in theforeground on the device) to reflect the new status. Illustratively, ifthe user is currently viewing the conversation with the user of device102, she may see a status indicator (e.g., in the header of the display)change to “online” or “idle” or “writing” or something else.

In operation 520, device 102 uploads the audio recording tocommunication system 120 (e.g., to a media server). The device may alsosend any text that the user of device 102 entered before and/or aftermaking the recording. The recording may be transcoded, compressed and/orotherwise processed before or after it is uploaded.

In operation 522, system 120 stores the recording, reports it or offersit to device 104, and delivers any text that may have accompanied therecording. In some implementations, the system may attempt toimmediately deliver the recording if device 104 is online. In otherimplementations, the system may deliver a message that appears in theconversation displayed on device 104, which may include any textassociated with the recording, and that enables the user of device 104to initiate a download of the recording. When the recording isdownloaded to device 104, a control for initiating its playback isinserted into the conversation. Thus, the user of device 104 cancommence playback by pressing one icon or control.

In operation 530, the user commences playback of the recording on device104. The communication application therefore sends an ephemeral signalto system 120 to report this change in status.

In operation 532, the system receives the notification, processes itsimilar to the notifications from device 102, and relays it to device102.

In operation 534, device 102 receives the notification and updates astatus of the users' conversation if possible (e.g., if the applicationis executing in the foreground on device 102). For example, a portion ofan identifier or thumbnail representing the user of device 104 (e.g., anicon) may change color, may change to a different shape, a check-mark orother symbol may be displayed, etc.

In optional operation 540, the user of device 104 moves device closer toher or further away from her during playback of the recording, and theplayback proceeds through different output components. For example,while the device is relatively far from her a first speaker may beused—such as a speaker that is activated when the device is placed in aspeakerphone mode of operation. When the device is placed close to theuser (e.g., as if she were making a call with it), audio output mayautomatically switch to a speaker used when she is making a call and isnot in speakerphone mode.

In optional operation 550, the user of device 102 creates a text messageto the user of device 104, such as a follow-up message to the audiorecording, and sends it using the same multi-function tool that wasactivated to create and send the audio recording. Thus, in someembodiments of the invention, a single context-sensitive control is usedto send both text and recorded audio within a communication session.

FIG. 6 is a block diagram of a portable device for conducting recordedaudio communications with real-time status notifications, according tosome embodiments of the invention.

Device 602 comprises one or more processing units or processors 602,touch-screen display 604, soft and/or hard input controls 606 (e.g.,keypad, keyboard, mouse, trackball), secondary storage 608, wirelesstransceiver 610 (coupled to antenna 610 a), optional additionalinput/output ports 612 and memory 618, all coupled via bus 614. Wirelesstransceiver 610 may have a persistent communication connection to awireless communication service provider (e.g., a wireless telephonecompany's network).

Memory 618 stores programs and/or other instructions for execution byprocessor(s) 602. These programs include operating system 630 formanaging basic device services and performing hardware-dependentfunctions and tasks.

Memory 618 also stores multiple program modules or sets ofprocessor-executable instructions that support and/or implement methodsof exchanging voice communications with real-time notifications. Inparticular, memory 618 includes communication program 620 for conductingreal-time communications via text and/or media. Program 620 may beprovided by the operator of a communication system such as system 120 ofFIG. 1.

As part of communication application 620, or in addition to theapplication, memory 618 stores graphical user interface 622,notification logic 624, communications 626 (which include audiorecordings 628), etc.

GUI 622 displays and manages various contents on touch-screen display604, including a multi-function control for initiating one or both of arecorded audio message and a textual message, and which may changeappearance in different contexts to signal its functionality. GUI 622further includes a communication status indicator that changesappearance when an associated audio recording is played, and which mayfurther change appearance for other reasons (e.g., a recipient forwardsthe audio recording or deletes it). Yet further, GUI 622 includes astatus of a communication partner (e.g., to indicate when that partneris recording audio), representations of communications to display in aconversation region of display 604, thumbnail images of communicationpartners, and so on.

Notification logic 624, when executed by processor(s) 602 issues and/orconsumes real-time notifications. For example, logic 624 may issue anotification toward another device when an operator of device 602records audio to be sent to a user of the other device, may send anothernotification when recording is complete, and another when the operatorof device 602 plays a recording received from the other device.

Communications 626 include at least some of the communications exchangedbetween the operator of device 602 and participants in the operator'scommunication/chat sessions. Other (e.g., older) communications may bestored in secondary storage 608 until needed (e.g., if the deviceoperator scrolls through his communications).

An environment in which some embodiments of the invention are executedmay incorporate a general-purpose computer or a special-purpose devicesuch as a hand-held computer or communication device. Some details ofsuch devices (e.g., processor, memory, data storage, display) may beomitted for the sake of clarity. A component such as a processor ormemory to which one or more tasks or functions are attributed may be ageneral component temporarily configured to perform the specified taskor function, or may be a specific component manufactured to perform thetask or function. The term “processor” as used herein refers to one ormore electronic circuits, devices, chips, processing cores and/or othercomponents configured to process data and/or computer program code.

Data structures and program code described in this detailed descriptionare typically stored on a non-transitory computer-readable storagemedium, which may be any device or medium that can store code and/ordata for use by a computer system. Non-transitory computer-readablestorage media include, but are not limited to, volatile memory,non-volatile memory, magnetic and optical storage devices such as diskdrives, magnetic tape, CDs (compact discs) and DVDs (digital versatilediscs or digital video discs), solid-state drives and/or othernon-transitory computer-readable media now known or later developed.

Methods and processes described in the detailed description can beembodied as code and/or data, which can be stored in a non-transitorycomputer-readable storage medium as described above. When a processor orcomputer system reads and executes the code and manipulates the datastored on the medium, the processor or computer system performs themethods and processes embodied as code and data structures and storedwithin the medium.

The foregoing descriptions of embodiments of the invention have beenpresented for purposes of illustration and description only. They arenot intended to be exhaustive or to limit the invention to the formsdisclosed. Accordingly, many modifications and variations will beapparent to practitioners skilled in the art. The scope of the inventionis defined by the appended claims, not the preceding disclosure.

1. A method of providing real-time notifications of audio communication status, the method comprising: commencing an audio recording at a first device; transmitting, using a processor of the first device, a first notification transmitted after commencement of the audio recording at the first device and during recording of the audio at the first device, the notification configured to cause a second device to display a status of the audio recording in response to receiving the first notification; completing the audio recording at the first device; and transmitting the audio recording toward the second device.
 2. The method of claim 1, wherein the first notification is transmitted to an intermediate device and is configured to be relayed to the second device in real-time.
 3. The method of claim 1, wherein the first notification is transmitted to an intermediate device and is configured to be relayed to the second device during recording of the audio at the first device.
 4. The method of claim 1, further comprising: receiving a second notification from the second device, and updating a display at the first device, the display indicating that the second device has commenced playback of the audio recording, wherein the second notification is received from the second device in real-time.
 5. The method of claim 1, further comprising: receiving a second notification from the second device, and updating a display at the first device, the display indicating that the second device has commenced playback of the audio recording, wherein the second notification is received from the second device during playback of the audio recording at the second device.
 6. The method of claim 1, further comprising: transmitting the audio recording from the first device to an intermediate device, the transmission of the audio recording configured to cause the intermediate device to transmit a textual message to the second device, wherein the textual message comprises a link associated with the audio recording.
 7. The method of claim 1, further comprising: transmitting from the first device a third notification transmitted upon termination of the audio recording at the first device the third notification configured to cause the second device to remove the status of the audio recording.
 8. The method of claim 7, wherein removing the status of the audio recording comprises replacing the status of the audio recording with a status of the first device that was displayed on the second device before the status of the audio recording was displayed.
 9. The method of claim 1, wherein the status of the audio recording displayed on the second device is a textual message.
 10. The method of claim 4, wherein changing the display of the status of the second device in response to the second notification comprises altering an appearance of a graphical representation associated with a status of the audio recording at the second device.
 11. The method of claim 5, wherein the first device changes the display of the status of the second device before playback of the audio recording at the second device has completed.
 12. The method of claim 1, wherein: transmission of the first notification from the first device is triggered by activation of a control in a first mode by a user of the first device; and activation of the control in a second mode by the user triggers transmission of a textual message.
 13. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to: commence an audio recording at a first device; transmit, using a processor of the first device, a first notification transmitted after commencement of the audio recording at the first device and during recording of the audio at the first device, the notification configured to cause a second device to display a status of the audio recording in response to receiving the first notification; complete the audio recording at the first device; and transmit the audio recording toward the second device.
 14. A communication system for providing real-time notifications of audio communication status, the system comprising: a communication server comprising a first processor and a first memory storing instructions that, when executed by the first processor, cause the communication server to: receive from a first communication device a first notification transmitted upon commencement of an audio recording at the first communication device and before the first device has completed recording of the audio; and relay the first notification toward a second communication device during the recording of the audio, wherein the second communication device displays a status of the audio recording in response to the first notification; and a media server comprising a second processor and a second memory storing instructions that, when executed by the second processor, cause the media server to: store the audio recording; receive from the second communication device a signal identifying the audio recording; and in response to the signal, transmit the audio recording toward the second communication device.
 15. The communication system of claim 14, wherein the first communication device and the second communication device are engaged in a real-time communication session.
 16. The communication system of claim 15, wherein the communication session comprises an exchange of textual messages before, after, or before and after the audio recording.
 17. The communication system of claim 14, wherein the first memory further stores instructions that, when executed by the first processor, cause the communication server to: transmit the first notification toward a third communication device, wherein the third communication device displays a status of the audio recording in response to the first notification; wherein the first communication device, the second communication device and the third communication device are engaged in a group real-time communication session.
 18. The communication system of claim 17, wherein the first memory further stores instructions that, when executed by the first processor, cause the communication server to: receive from the third communication device a fourth notification transmitted upon commencement of playback of the audio recording at the third communication device; and transmit the fourth notification toward the first communication device, wherein the first communication device changes a display of a status of the third communication device in response to the fourth notification.
 19. An apparatus for providing real-time notifications of audio communication status, the apparatus comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the apparatus to: commence an audio recording at a first wireless communication device; transmit, using the one or more processors, a first notification transmitted after commencement of the audio recording at the first wireless communication device and during recording of the audio at the first wireless communication device, the notification configured to cause a second wireless communication device to display a status of the audio recording in response to receiving the first notification; complete the audio recording at the first wireless communication device; and transmit the audio recording toward the second wireless communication device. 